Vision-based computation of three-finger grasps on unknown planar objects

This paper presents an implemented vision-based strategy for computing three-finger stable grasps on unknown planar objects. Its only input is an image of a real unknown curved object instead of synthetic polygonal models. Grasp regions are identified for achieving tolerance around contact points so that errors in finger positioning as well as a finite finger size are considered. It can find expanding and squeezing grasps involving internal and external regions, including holes. Theoretical constraints for force-closure are met while assuring robustness and real-time performance under real-world conditions. 1 I n t r o d u c t i o n Extensive research in robotic grasping and dexterous manipulation over the last twenty years has established a theoretical framework for grasp analysis, simulation and synthesis [1, 2, 13]. However, there remains a gap between theoretical promise and practical delivery [13]. Indeed, in a survey including some 64 articles [20] Shimoga concludes that none of the synthesis algorithms have been implemented in real time thus far. Among the reasons for this situation, prohibitive computational complexity, lack of precise sensing capabilities and, importantly, the unstructuredness of real-life objects are often highlighted. These model-based theoretical approaches have dominated the field, [1, 2, 13, 20, 15]. Most algorithms rely on some critical assumptions, such as the availability of a complete geometric model of the object to be manipulated. Similarly, it is generally assumed in the literature that all contacts are point contacts as opposed to real finite fingers (enclosed by a rubber covering), and that friction coefficients (including torsional for soft fingers)are known a priori. The curvature of the object and the effector at the contact point has a significant effect on grasp stability [9] usually ignored in polygonal models. In unstructured service scenarios those assumptions are no longer reasonable and the use of vision and tactile sensing for dealing with essentially unknown objects is a must. Since there will be unavoidable errors in positioning and orienting the end-effectors, it is important to choose a grasp so that the system performance is insensitive to these positioning errors. This paper presents results within the framework of a larger project in collaboration with the University of Massachusetts [11] aimed at endowing a complete humanoid hand-eye system with the ability of robustly performing visually-guided precision planar grips. The system is composed of a stereo head and a three-finger hand with tactile sensing, it is able of (1)analyzing an image of an unknown object and identifying triplets of candidate grasp points; (2)selecting an adequate finger configuration, and (3)executing the grasp by correcting position errors by means of tactile sensing. Our motivation is to contribute to filling the gap between the well-founded theoretical framework and its practical delivery. This is by no means just an application problem, since original research methods are called for to overcome the inherent difficulties and assumptions of modelbased approaches, so that the desired theoretical constraints are met while assuring robustness and realtime performance under real-world conditions. This paper focuses on problem (1) above, an algorithm is proposed for the determination of stable planar (2D) three-finger grasps taking as input an image of the -possibly curvedobject to be grasped. The strategy takes into account real-world constraints such as finite contacts, finger position errors and uncertainty about friction coefficients. Our solution to problem (2) can be found in an accompanying paper [11]. Similarly, the approach to problem (3) is fully described in [16]. Vision-based approaches to two-finger grasping has been studied and widely developed in [4, 6, 7, 8, 10, 18] and [21]. However, in the case of threeii iiiiiil i iii iii i ii !!!iiiiiiii i iill iil i iiiii iiiiiii iii~ii~ i i i i ~iiiiiii!!iiiiii!iiiiii!~i!!ii~i~ii!~iii~ii!!!~!ii~iii!iiiiiiiiiiii~i~i~i~i~i~i~i~i~i!~iiiiiii~iiii iiiiiii!!iiiiiiiiiii~iii!~!iiii~iiiiii~iii~iiiii~!i~iiiiiii~iiiii~!~!iiii~iiiiiiii!!!~i~iiiiiiiiiiiiiiii!ii~iii~i~iii~iiiiii~i~iiiiii! ~ ~~i~ ii ii ii !iii ii ii!ii!!iiiil :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: ?) i !:i:!:~:i:i:! !8! i ! i ~:i:7::~-~:::i::i:!:i:i:!:i:i:i:i:i:!:i 8 i:i:i:i i:i:i!: i:i: ! ~!~!~ii~!ii~i~i~iii~i~iiiii~ii~!i~ii~iii~ii~i~!~i!~!~;~:.::~ ~ :.i: ;:~i~!~i~iii~iiiii~i~iiiiiii~iiii~ii~iii~!~iiiiiii~iiii~i~i~:~:~:~:i~i ............. ~,',',',::~::~::::::i~ii!!!i!,:,':: i i~,iiiii :ii,,~,i~ , ,,,,ii~iiy ~ i~iiiiiiiii ~i~i~i~i~iii~i~i ', , i ~!!,~,~,,~,~,, ,,,~, , ,~,,,,,i,,i,,ii,,,,,,iiiiii i! i!!?!~!!:i .................. ~ i ! i ! ! ! i ! i ! ~ i !!! ~ i i i ~ i~ ~!!iiiiiiiii!iiiiiiiiiiiiiiiiiiiiii) Figure 1" Three-finger ~ d.o.f robotic hand. Courtesy of Barrett Technology Inc. finger grasps, though there have been some important model-based theoretical contributions dealing with force closure in polygonal shapes [14, 17], there has been little, if any, implementation using as input real images of objects and including real-world constraints. This paper is organized as follows. First the theoretical foundations of the strategy are described; section 3 presents the proposed approach, and our experimental results are reported in section 4. 2 G r a s p C h a r a c t e r i z a t i o n For grasp determination, we assume finite contacts with friction between the object and the fingers, as well as the Coulomb friction model [12]. However, the static friction coefficient between the object and the fingers is not known beforehand. In order to establish whether a grasp is stable, two aspects must be taken into account: first, the local contact of each finger on the curved object contour and second, the set of forces generated by the three fingers on the object. For evaluating these two aspects we define two criteria. 2.1 Finger adaptation criterion. The stability of a contact point is directly related to the curvature of the surface at that point [9]. Considering this fact we establish the f inger adaptat ion criterion. It evaluates the adaptation between the gripper's fingers and the object in terms of the estimated area of contact when the fingers are placed on the object. This estimation is based on the curvature of the object's observed contour at the points where the fingers should be placed. This curvature should not exceed a threshold a (curvature threshold). In this way, we are assuring that two important practical requirements are met: first, a good finger-object contact surface and second, tolerance to position errors by selecting long enough grasp regions. iil: